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The subjects of the paper are the likelihood method (LM) and the expected Fisher information (FI) considered from 
the point od view of the construction of the physical models which originate in the statistical description of phenomena. 
The master equation case and structural information principle are derived. Then, the phenomenological description of 
the information transfer is presented. The extreme physical information (EPI) method is reviewed. As if marginal, the 
statistical interpretation of the amplitude of the system is given. The formalism developed in this paper would be also 
applied in quantum information processing and quantum game theory. 
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1 Introduction. The subjects ofthe paper are the like- 
lihood method (LM) and the expected Fisher information 
(FI), I, considered from the point of view of the construc- 
tion of the physical models, which originate in the statisti- 
cal description of phenomena. The FI had been introduced 
by Fisher HI as a part of the new technique of parameter 
estimation of models which undergo the statistical investi- 
gation in the language of the maximization of the value of 
the likelihood function L (signed below as P). The notion 
of the FI is also connected with the Cramer-Rao inequal- 
ity, being the maximal inverse of the possible value of the 
mean-square error of the unbiased estimate from the true 
value of the parameter The set — {On) of the parame- 
ter£] composes the components in the statistical space S. 
Under the regularity conditions the expected FI (here be- 



In the most general case of the estimation procedure the di- 
mensions of 6* = {Oi)\ and the sample y = {yn)\ are usually 
different, yet because (6i) is below the set of physical "positions" 
of a physical nature (e.g. positions in the case of an equation of 
motion or energies in the case of a statistical physics generating 
equation 0) we have -E(yn) = On and the dimensions of O and 
y are the same, i.e. k = N. 



cause of the summation over n, the channel capacity): 



N 



n=l 

N 
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(1) 



is the variance of the score function 8(6*) = dlnP{0) / 80 
||2l. It describes the local properties of P(y; Oi, ...jOn) 
which is the joint probability (density) of the N data values 
y = (yi, ...,yi^), but is understood as a function of the 
parameters 6'„. Below with x„ = y„ — 6'„ we note the 
added fluctuations of the data y„ from 6'„. 

The development of the statistical methods introduced 
by Fisher H] has followed along two different routes. The 
first one, of the differential geometry origin, began when 
C.R. Rao m noticed that the FI matrix defines a Rieman- 
nian metric. For a short review of the following work, until 
the consistent definition of the a-connection on the statis- 
tical spaces see [5|. The second one is based on the con- 
struction of the information (entropical) principles and we 
will describe this approach in details. It was put forward 
by Frieden and his coauthors, especially Soffer |[3l, and is 
known under the notion of the extreme physical informa- 



Copyright line will be provided by the publisher 



2 



Piotrowski, Sladkowski, Syska, Zajgc: Likelihood method and Fisher information in construction of physical models 



tion (EPI). Frieden began the construction of the physical 
models by obtaining the kinetic term from the Ffl 
Then he postulated the variational (scalar) information prin- 
ciple (IP) and internal (structural) one. Each of the IP has 
the universal form, yet its realization depends on the partic- 
ular phenomenon. The variational IP leads to the dispersion 
relation proper for the system. The structural IP describes 
the inner characteristics of the system and enabled [3j the 
fixing of the relation between the FI (the channel capac- 
ity) / and the structural information (SI) Q f]\. The inter- 
esting point is that a lot of calculations is performed when 
the so called physical information K is partitioned equally 
into / and Q (or with the factor 1/2), having the total value 
equal to zero. The method is fruitful as Frieden derived 



Let for a system the square distance between two of its states 
denoted by q(x) and g'(x) = (g + ciq)(x) could be written in the 
Euclidean form ds^ = dx dq dq where X is the space of the 
positions x (in one measurement). Supposing that the states are 
described by the probability distributions we compare it with the 
distance ds^ ~ 1 Ix '^■^ p^x f space 5 with the Fisher- 

Rao metric (6), where p(x) and p' = {p + dp)(x) are the proba- 
biUty distributions for these two states. Then we notice that these 
two formulas for ds^ coincide if g(x) = \/p(x). In this way 
the notion of the amplitude g(x) appears naturally from the 
Riemannian, Fisher-Rao metric of the space <S. It satisfies the 
condition: dxp(x) = dxg(x)g(x) — 1 . 
Now, let the sample of dimension A'^ be "collected" by the system 
and qn are the real amplitudes and pa;„(x„) are the probability 
distributions with the property of the shift invariance Pa;„(x„) 
= Pi„(x„|6l„) = p„{yn\On) where x„ = - 6„. Assuming 
that the data are collected independently which gives the factor- 
ization property P{y) = P{y\0) = nn=iP"(y"l^")' where 
6,n has no influence on y„ for m n, and using the chain rule 
d/de„ = d/diyn-On) a(yn-6i„)/96i„ = - d/d{yr.-dn) = 
— (?/9xn, the transition from the statistical form of FI given by 
Eq.^TJ to its kinematical form given below by the first possibility: 



n=l"' ^ ^ n = l 



9x 



9x 



(2) 



is performed (3). The second possibility is achieved |[3] as fol- 
lows: Using the amplitudes g„, the wave function i/j„ could 



be constructed as il>„ 



^(g2n-l + iq2n), whcrC U = 



Vn 



1, 2, N/2 with the number of the real degrees of freedom be- 
ing twice the complex ones. With this, the first kinematical form 
of I in Eq.l[2ll could be rewritten in the second form, where ad- 
ditionally the index n has been dropped from the integral as the 
range of all x„ is the same. Finally, the total probability law for 
all data gives p(x) = J^'^^i P^^ (x„|6»„)P(6'„) = ^ Y.Z=\ <&■' 
where we have chosen P{9n) = -^(31. All of these lead to 

= V'^V'n. With the shift invariance condition and 

the factorization property, I does not depend on the parameter set 
(d-n) 12, and the wave functions ipn do not depend on these pa- 
rameters, i.e. (expected) positions, also ID. From this summary 
of the Frieden method we notice that although the wave functions 
ipn are complex the quantum mechanics is really the statistical 
method. 



(i) the Klein-Gordon eq. for the field with the rank N (for 
the scalar field N = 2), (ii) the Dirac eq. for the spinorial 
field with iV = 8, (iii) the Rarita-Schwinger eq. for higher 
N and (iv) the Maxwell eqs. for TV = 4. The generality of 
the method enables to describe the general relativity theory 
and take into account gauge symmetry also f3l. Using the 
information principles (IPs), Frieden gave the information 
theory based method of obtaining the Heisenberg principle 
which from the statistical point of view is the kind of the 
Cramer-Rao theorem with the FI of the system written, 
after the Fourier transformation, in the momentum repre- 
sentatiorQ. Then the formalism enabled both to obtain the 
upper bound for the rate of the changing of the entropy of 
the analyzed system and to derive the classical statistical 
physics also fS). 

In our paper we develop the less informational approach 
to the construction of the IP using the notion of the total 
physical information (TP I), if = Q + / Q, instead of 
the change of the physical information K = / — J in- 
troduced in 13]. This difference does not affect the deriva- 
tion of the equation of motion (or generating eq.) for the 
problems analyzed until now |3 |, yet the change of the no- 
tion of K to TPI with its partition into the kinematical 
and structural degrees of freedom, goes closely with the 
recent attempts of the construction of the principle of en- 
tropy (information) partition. In Section [Z2l we will derive 
the structural IP from the analysis of the Taylor expansion 
for the log-likelihood function. Our approach to the IP re- 
sults also in the change of the understanding of the essence 
of the information (entropy) transfer in the process of mea- 
surement when this transfer from the structural to the kine- 
matical degrees of freedom takes place. We will discuss it 
in Section [5] Despite the differences we still call this ap- 
proach the Frieden one and the method the EPI estimation. 
Finally, with the interpretation of / as the kinematical term 
of the theory, the statistical proof on the impossibility of 
the derivation of the wave mechanics and field theories for 
which the rank of the field is finite from the classical 
mechanics, was given Q. 

2 The master equations vs structural information 
principle The maximum Hkehhood method (MLM) is based 
on the set of N likelihood equations HI : 



\e=e 



d_ 



lnP{0) 



0=0~ 



0, 



(3) 



where the MLM's set of N estimators O = {6^)1 is its 
solution. These N conditions for the estimates maximize 
the likelihood of the sample. 

2.1 The master equations. Yet, we could approach 
to the estimation differently. So, after the Taylor expanding 
of ^(6*) around the true value of O and integrating over the 



' In reality the Fourier transformation forms the type of the 
entanglement between the variables of the system. 
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sample space we obtain: 

d^xfp(0) -P(0) 



N 



1 /12, 



E 



-'11 ^11 1 \^n' 



d0n 



') + 



9n) 

(4) 



where the notation P{0) = P{0) je has been used. When 
the integration over the whole sample space is performed 
then neglecting the higher order term^ and using the nor- 
malization condition / d^xP(6>) = J d^xP{0) = 1 
we see that the LHS of Eq.© is equal to zero. Hence we 
obtain the result which for the locally unbiased estimators 
IJl and after postulating the zeroing of the integrand at the 
RHS, takes for particular n and n' the following micro- 
scopic form of the master equation: 

f>a J i^n ^ en)ien' ~ On,) ^ . (5) 

When the parameter 6*^ has the Minkowskian index 1/ then 

P = nn=i Pni^n) ^'^'^ Eq.© Icads, after the transition to 
the Fisherian variables (as in Footnote 2), to the form of 
the equation of conservation of flow: 



E 



dxl. 







0° - 9" 



(6) 



Here 6*^ and 6*° are the expected position 



dt„ 
where t„ 

and time of the system, respectively and index n could be 
omitted. 

2.2 The information principle. The EPI method. 

Using InP instead of P and after the Taylor expanding of 
InPiO) around the true value of and integrating with 
the d^x P{0) measure over the sample space we obtain 
(instead of Eq.(|4|i) the equation of the EPI method of the 
model estimation: 



d'^^P{0){ln 



P{0) 



N 



^3-E 



dlnP{0) 
d9n 



{On 



N 



n,n' — 1 



0) 



(7) 



with P{0) EE P{0) \e. The term on the LHS of Eq.© 
has the form of the modified relative entropy. Let us define 
the (observed) structure tF of the system as follows: 



When we define Q as 



d'^xP{0) tF 



^ dlnP 



^ d9n 



(8) 



(9) 



'* Which is exact even at the density level if only P{0) € S has 
not higher than the 2-order jets at 0. 



then we obtain the structural equation of the IP form: 
-Q = i (10) 



\jd-.Pi0)j: 



dHnP 

d9n'd9n 



){9n — 9n){9n' ~ On')- 



Now, let us postulate the validity of Eq.dTOl) on the micro- 
scopic level: 

TV N 
n—1 n—1 



TV 

^ ^ ^ 7in' {^n ~ ^n 
n,n' — 1 



)i9n' - 9n') = Arms ■ (H) 



As the inverse of the covariance matrix the (observed) Fisher 
information matrix: 



iF= - 



d^lnP{0)\ 

d9n'den J 



(12) 



is symmetric and positively defined. It follows that there 
is an orthogonal matrix U such that Arms in Eq.lfTTI) and 
hence A lhs also, could be written in the normal form: 



N 



iF„„' {9n ~ 9n){9n' — ^n') 

n,n' — l 



N 



Arms = Alms = ^ mnvl , (13) 



where u„ are some functions of ^„-s and m„ are the ele- 
ments (obtained for Alhs) of the positive diagonal matrix 
itF which because of the equality (fTsT l has to be equal to the 
diagonal matrix obtained for Arhs, i-e.: 



r:F = D'^ U'^ iF U D 



(14) 



Here D is the scaling diagonal matrix with the elements 



dn = 



and A„-s are the eigenvalues of iF. Finally, we 



could rewrite Eq.(fT4]i in the form of the Frieden structural 
microscopic IP: 



0, 



where 



(15) 



(16) 



The existence of the normal form ( fT3] ) is a very strong 
condition which makes the whole Frieden analysis possi- 
ble. Two particular assumptions lead to the simple physical 
cases. When the structure tF = then from Eq. lfTTT i we ob- 
tain a form of the "master equation" (compare with Eq.©) 
with: 



mF = diag 2 



dlnP 

'de7 



9n -9n, tF = (17) 
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and dn ~ ^2 / A„ . If we instead suppose that the 
distribution is regular ^ then with 



dbiP 



for all n 



1, N, we see from Eq.lfTTli that (d„ = ^2/A„): 

After integrating Eq.dTSb with the measure d'^xPiO) we 
recover the integral structural IP (postulated previously in 
131 although in a different form and interpretation but) in 
exactly the same form as in Q: 

+ / = , (19) 
where / is the Fisher information channel capacity: 



N 

1 = J d^xP{0) (it) 

n,n' — l 



(20) 



and Q is the SI: 



N 



Q = J d^xP{0) Y ■ (21) 

n,n' — l 

The IP given by Eq.(fT9l) is the structural equation of many 
current physical models. 

3 The information transfer As / is the infinitesimal 
type of the Kulback-Leibler entropy 13] which in the sta- 
tistical estimation is used as a tool in the model choosing 
procedure, hence the conjecture appears that / could be 
(after imposing the variational and structural IPs) the cor- 
nerstone of the equation of motion (or generating equation) 
of the physical system. These equations are to be the best 
from the point of view of the IPs what is the essence of the 
Frieden's EPI. The inner statistical thought in the Frieden 
method is that the probing (sampling) of the space-time 
by the system (even when not subjected to the real mea- 
surement) is performed by the system alone, that using its 
proper field (and connected amplitudes) of rank N, which 
is the size of the sample, probes with its kinematical "Fish- 
erian" degrees of freedom the position space accessible for 
it. The transition from the statistical form ([T]) of the FI to 
its kinematical representations ^ is given in Footnote 2. 
Let us consider the following informational scheme of the 
system. Before the measurement takes place the system 
has / of the system which is contained in the kinemati- 
cal degrees of freedom and Q of the system contained in 
the structural degrees of freedom, as in Figure la. Now, let 
us "switch on" the measurement during which the transfer 
of the information (TI) follows the rules (see Figure lb): 

J > hence 51 = I'- I >Q , 5Q ^ Q' - Q < 0, (22) 

where /', Q' are the FI and SI after the measurement, 
respectively and J is the TI. We postulate that in the mea- 
surement the TI is ideal at the "point" q, i.e. we have that 
Q = Q' + J= Q + 5Q + J hence SQ = — J. This means 



that the whole change of the SI at the "point" q is trans- 
ferred. On the other hand at the "point" / the rule for the 
TI is that I' < I + J hence < SI = I' - I < J. 
Therefore, as J > we obtain that \6I\ < \SQ\, the re- 
sult which is sensible as in the measurement information 
might be lost. In the ideal measurement we have obtained 
that 6Q = -51. 

Previously we postulated the existence of the additive total 
physical information (TP I) Q: 



K = 



(23) 



In Is) the intuitive condition that K > was chosen lead- 
ing to the following form of the structural IP: 



kQ+I =0 



(24) 



Q + I = for K^l , (25) 

which we now derived in Eq.(fT9b. In the case of Eq.(l25]l 
we obtain that K is equal to 



K = 



1 = for K=l 



(26) 



The structural (internal) IP in Eq. (l24l) l?) is operationally 
equivalent to the one postulated by Frieden |0, hence it has 
at least the same predictive power. For the short descrip- 
tion of the difference between both approaches see Q- The 
other IP is the scalar (variational) one. It has the form Q: 

6K = 6{Q + /) = => K = Q + I is extremal (27) 

The principles ( l24b and dZTl i are the cornerstone of the 
Frieden EPI. They form the set of two differential equa- 
tions for the amplitudes (/„ which could be consistent, lead- 
ing for K = 1 or 1 /2 to many well known models of the 
field theory and statistical physics [Sl, as we mentioned in 
the Introduction. It could be instructive to recalculate them 
again using the new interpretation of if H]. 
Finally, let us notice that in the Concorde with the postu- 
lated behavior of the system in the measurement, we have 
obtained SI < J = —6Q from which it follows that: 

K" = 1' + Q' < {I + J) + {Q-J)=I + Q = K 

K' < K . (28) 

In the case of 51 = —SQ we obtain K' = K. Then it 
means that the TPI remains unchanged in the ideal mea- 
surement {51 = —5Q) which, if performed on the intrinsic 
level of sampling the space by the system alone (and not 
by the observer), could possibly lead to the variational IP 
given by Eq.(l27b. 

4 Conclusions The overwhelming impression of the 
EPI is that the TPI is the ancestor of the Lagrangian Q. 
As the Fisherian statistical formalism and hence the IPs 
lie here really as the background of the description of the 
physical system it is not the matter of interpretation only 
especially that the structural IP (which is very close to 
the approach used in llllll) was derived in Section IZ21 In 
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Figure 1 Panel: (a) The system before the measurement: Q is the SI of the system contained in the structural degrees 
of freedom and / is the FI of the system contained in the kinematical degrees of freedom, (b) The system after the 
measurement: Q' is the SI and /' is the FI of the system; 5Q ^ Q' — Q < and 5/ = /' — /> as the transfer of the 
information (TI) in the measurement takes place with J > 0. In the ideal measurement SI = —5Q. 



the result, we have proved that the whole work of Frieden 
with our equation of motion (or generating equation) ( fTOl i 
lie inside the same EPI type of the statistical modification 
of LM. In the contrary the master equation (01) lies in the 
other kind of the classical statistics estimation, closer to 
the stochastic processes, than the EPI method. Besides, for 
the exponential models with quadratic {k = 2) Taylor ex- 
pansion, the simple microscopic forms ( fTTI i and ( fTSl l are 
also given in Section |272l Yet e.g. for the family of the dis- 
tributions with k > 2 and for the mixture family models 
151 the further investigations are needed. Here the alterna- 
tive way of solving the IPs equations ( l24b and ( |27] i given 
by Frieden seems more appealing, as besides the boundary 
conditions, it is not restricted to the particular shape of the 
distributions. 

The presented version of the EPI also has, as the original 
one in has, the ability of the synonymous description of 
the unmeasured and measured system, but additionally it 
is characterized by the more precise distinction of these 
two situations [S]- Is is connected with the meaning of K 
as the TPI which enables the precise discrimination of the 
inner "measurement" performed by the system alone from 
the one performed by an observer. Next, the chosen form 
of the internal IP ( |24| ). suggests entanglement of the obser- 
vational space of the data with the whole unobserved space 
of positions of the system [S). This situation is also a little 
bit different than in JH where (also because of the inter- 
pretation of K) the entanglement with the part of the un- 
observed configuration only was seen. E.g. in the descrip- 
tion of the EPR-Bohm effect our approach describes the 
entanglement which takes place between the projection of 
the spin of the observed particle and the unobserved joint 
configuration of the system [S]. Yet, the shared feature is 
that (jfl does not represent the SI of the system only but 
the information on the entanglement seen in the correla- 
tions of the data in the measurement space also. Therefore 
in general the EPI could be used as the tool in the estima- 
tion of the entangled states. Our result is that the source of 
the (self)entanglement could be understood as the conse- 
quence of the partition of the Taylor expansion of the log- 
likelihood. Now, for further development of the EPI the 
differential geometry language H of the IPs is needed. 



^ E.g. in (3) the scalar field case is analyzed for which Q is 
connected with the rest mass of the particle. 



Finally, let us at least mention that there are other fields 
of science were EPI could be used, e.g. the econophysics 
ll3l l9lfT0l . We envisage that the formalism developed in this 
paper would be applied in quantum information processing 
and quantum game theory also 191 1101 . 

Acknowledgements This work is supported by L.J.CH.. 
This work was supported in part by the scientific network Labora- 
torium Fizycznych Podstaw Przetwarzcinia Informacji sponsored 
by the Polish Ministry of Science and Higher Education (deci- 
sion no 106/E-345/BWSN-0 166/2008) and by the University of 
Silesia grant. 

References 

[1] R.A. Fisher, Phil. Trans. R. Soc. Lond. 222, 309 (1922). 
R.A. Fisher, Statistical methods and scientific inference, 2nd 
edn. (London, Oliver and Boyd, 1959). 

[2] Y. Pawitan, In all likelihood: Statistical modelling and infer- 
ence using likelihood, (Oxford Univ. Press, 2001). 

[3] B.R. Frieden, Found. Phys.l6, 883 (1986); Phys. Rev A 41, 
4265 (1990). B.R. Frieden, B.H. Soffer, Phys. Rev. E 52, 
2247 (1995). B.R. Frieden, Phys. Rev A 66, 022107 (2002). 
B.R. Frieden, A. Plastino, A.R. Plastino and B.H. Soffer, 
Phys. Rev. E 66, 046128 (2002); Phys. Lett. A 304, 73 
(2002). B.R. Frieden, Science from Fisher information: A 
unification, (Cambridge Univ. Press, 2004). 

[4] C.R. Rao, Bulletin of the Calcutta Math. Soc. 37, 81-91 
(1945). 

[5]B. Efron, Ann. Statistics 3, 1189 (1975). A.R Dawid, 
Ann. Statistics 3, 1231 (1975); Ann.Statistics 5, 1249 
(1977). S. Amari, Ann. Statistics 10, 357 (1982). H. Na- 
gaoka and S. Amari, Technical Report METR 82-7, Dept. 
of Math. Eng. and Instr. Phys, Univ. of Tokyo, (1982). A. 
Fujiwara and H. Nagaoka, Phys. Lett. A 201, 119 (1995); 
J. Math. Phys. 40, 4227 (1999). K. Matsumoto, PhD thesis, 
Univ. of Tokyo, (1998). S. Amari, H. Nagaoka, Methods of 
information geometry, (Oxford Univ. Press, 2000). 

[6] I. Bengtsson, K. Zyczkowski, Geometry of quantum states, 
(Cambridge Univ. Press, 2006). 

[7] J. Syska, Phys. Stat. Sol.(b), 244, No.7, 2531-2537 
(2007)/DOI 10. 1002/pssb. 200674646. 

[8] D. Mroziakiewicz, Master thesis, (in Polish) unpublished, 
Inst, of Physics, Univ. of Silesia, Poland, (2008). 

[9] E. W. Piotrowski, J. Sladkowski, Int. J. Theor. Phys. 42, 
1089 (2003). 



Copyright line will be provided by the publisher 



6 Piotrowski, Sladkowski, Syska, Zajgc: Likelihood method and Fisher information in construction of physical models 



[10] E. W. Piotrowski, J. Sladkowski and J. Syska, presented at 
ttie SIGMAPHI 2008 conference, to appear in Central Eu- 
ropean Journal of Physics. 



Copyright line will be provided by the publisher 



